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We report on a search for charge 1/3 third generation leptoquarks (LQ) produced in pp collisions 
at y/s — 1.8 TeV using the D0 detector at Fermilab. Third generation leptoquarks are assumed to 
be produced in pairs and to decay to a tau neutrino and a b quark with branching fraction B. We 
place upper limits on o~(pp — > LQ LQ) ■ B as a function of the leptoquark mass Mlq- Assuming 
B = 1, we exclude at the 95% confidence level third generation scalar leptoquarks with Mlq < 94 
GeV/c 2 , and third generation vector leptoquarks with Mlq < 216 GeV/c 2 (Mlq < 148 GeV/c 2 ) 
assuming Yang-Mills (anomalous) coupling. 
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Leptoquarks (LQ) are bosons predicted in many ex- 
tensions to the standard model . They carry both lep- 
ton and color quantum numbers, couple to leptons and 
quarks, and decay via LQ — > I + q. To satisfy experi- 
mental constraints on flavor changing neutral currents, 
leptoquarks of mass accessible to current collider exper- 
iments are constrained to couple to only one generation 
of leptons and quarks ||. Therefore, only leptoquarks 
which couple within a single generation are considered 
here. 

This Letter reports the results of a search for charge 
1/3 third generation leptoquarks produced in pp colli- 
sions at y/s — 1.8 TeV. We assume that leptoquarks 
are produced in pairs by QCD processes such as pp — > 
g — > LQ LQ + X. This process dominates over other 
production mechanisms which depend on the unknown 
leptoquark-lepton-quark coupling A under the standard 
condition A < V^tqem- We search for the decay sig- 
nature where both leptoquarks decay via LQ —ti> T -\-b 
resulting in a v T v T bb final state. For a leptoquark mass 
(Mlq) smaller than the mass of the top quark (mt), the 
decay LQ — > r + 1 is forbidden, and the branching frac- 
tion for the v T b mode, B, is unity For Mlq > m t , 
phase space factors suppress rt decays relative to the u T b 
channel. In this paper we give limits on the pair pro- 
duction cross section times B 2 (a ■ B 2 ) for Mlq between 
50 GeV/c 2 and 300 GeV/c 2 . Limits on the cross section 
are used to set limits on the third generation leptoquark 
mass for scalar and vector leptoquarks. Previous lim- 
its from the LEP e + e~ collider exclude all third genera- 
tion leptoquarks with masses below 45 GeV/c 2 [Q, while 
the CDF Collaboration has set limits for pair produced 
charge 2/3 or 4/3 third generation leptoquarks decaying 
via LQ -> r + b ||. 

The data used for this analysis were collected by the 
D0 detector || operating at the Fermilab Tevatron Col- 
lider during the 1993-1996 period. The D0 detector 
is composed of three major systems: an inner detector 
for tracking charged particles, a uranium/liquid argon 
calorimeter for measuring electromagnetic and hadronic 
showers, and a muon spectrometer consisting of a magne- 
tized iron toroid and three layers of drift tubes. The de- 
tector measures jets with an energy resolution of approxi- 
mately a /E — 0.8/ \[E (E in GeV) and muons with a mo- 
mentum resolution of a/p = [( °- 18( p p ~ 2) ) 2 + (O.OOip) 2 } 1 / 2 
(p in GeV/c). Missing transverse energy {lp T ) is deter- 
mined by summing the calorimeter and muon transverse 
energies, and is measured with a resolution of a = 1.08 
GeV + 0.019-(£|£t|) (E t in GeV). 

The decay of a b quark is indicated by the presence of 
a muon associated with a jet. We use three triggers to 
collect candidate leptoquark events, each requiring one 
or two muons A dimuon trigger required two muons 
with transverse momentum p T > 3.0 GeV/c. One single 
muon trigger required a muon with p T > 1.0 GeV/c and 



a jet with E T > 10 GeV. The other single muon trigger 
required a muon with p T > 10 GeV/c in the trigger and 
p T > 15 GeV/c during offline analysis, and a jet with 
E 3 T > 15 GeV. Integrated luminosities of 60.1 pb _1 , 19.5 
pb _1 , and 92.4 pb _1 respectively were collected using 
these three triggers. 

The offline analysis uses muons in the pseudorapid- 
ity range \rj^\ < 1.0 with p T > 3.5 GeV/c. The muon 
trajectories are required to be consistent with the re- 
constructed vertex position and have associated energy 
in the calorimeter. For events from either single muon 
trigger additional requirements are imposed: the pres- 
ence of hits in all three muon detector layers, a matching 
track in the central detector, and a good fit Q when 
these elements are combined. For the dimuon trigger 
events, at least one of the two muons must satisfy each 
of these additional requirements. Jets are reconstructed 
using only calorimeter energy with a cone algorithm of 
radius R = 0.7 where R = ^(A(f>) 2 + (Arj) 2 about the 
jet's centroid and 4> is the azimuthal angle. Each jet is 
required to have E J T > 10 GeV and to satisfy reconstruc- 
tion quality criteria ||. For the dimuon trigger, events 
with dimuon invariant mass greater than 8 GeV/c 2 are 
selected to eliminate backgrounds from low mass reso- 
nances and each muon must be associated with a differ- 
ent jet with Ai? M _j et < 0.5 to increase b quark purity 
ifjof . For the single muon triggers, we require a jet as- 
sociated with the muon with the same AR requirement 
plus an additional jet with E T > 25 GeV and \rjj\ < 1.5. 

We use Jfjrp to identify neutrinos. To help eliminate 
events with poorly measured $ T , we reject events where 
the azimuthal angular separation, A<p, between the miss- 
ing energy and the nearest jet is less than 0.7 radians. In 
some events, the source of the measured $ T is not mis- 
measured jets, but rather noise in individual calorimeter 
cells or from cells activated by the background generated 
by the Main Ring accelerator, which passes through the 
calorimeter. Such events are removed if the vector sum of 
the transverse energies in the jets and muons is consistent 
with zero. 

Data from the two single muon triggers with large 
$J T have contributions from W+ > 2 jet events where 
W — > \iv and the muon overlaps a jet. We use two vari- 
ables, and F M , for events passing those triggers to re- 
duce the W boson acceptance. We define = p T / Hlj\ 3 , 
where Hj? is the scalar sum of the Et of the jets and 
muons in the event. In Fig. [l] we compare the Z^ distri- 
butions of data events that pass the low-pr single muon 
trigger to Monte Carlo (MC) samples |ll[] satisfying a 
simulation of the trigger. There, for illustrative purposes, 
we select events with $ T > 30 GeV and A</> > 0.6 so that 
the data shown have roughly equal contributions from W 
boson and hadronic multijet events. Also shown are MC 
distributions of equal numbers of multijet events (which 
are b quark-dominated due to their muon content) and 
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FIG. 1. The distribution for events with $ T > 30 GeV 
and A<f) > 0.6 from (a) data compared to MC events from 
multiple jet (solid) and W — » ya> (dashed) processes and (b) 
leptoquark events with Mlq = 100 GeV/c 2 . The arrow indi- 
cates the requirement < 0.20 used in this analysis. 



W — ► \xv events. The data (normalized to the total num- 
ber of MC events) are consistent with the sum of these 
sources. Also shown is the same distribution for a lep- 
toquark MC sample with M l q = 100 GeV/c 2 satisfying 
the same criteria. Since the leptoquark distribution 
is determined by b quark decay kinematics, it is simi- 
lar to the multijet distribution. Requiring Z^ < 0.20 
eliminates about 90% of the remaining W — ► fj,v back- 
ground while maintaining a signal efficiency of 82% for 
M LQ = 100 GeV/c 2 . The second variable, F^, is the 
ratio of calorimeter energy within a cone of 0.4 centered 
on the muon's direction to that within a cone of 0.6: 
Ff, = E{R ll < 0A)/E(Rf, < 0.6). Most hadronic energy 
in higher Et direct b quark decays is spatially close to 
the muon. Requiring F^ > 0.80 removes about 84% of 
the W — > fiv background with a leptoquark signal effi- 
ciency of 82%. MC studies indicate that Z^ and F^ have 
little correlation. 

The data from the high-py muon trigger include a sig- 
nificant contribution from top quark pair production (ti) 
events. The scalar sum of jet Et, H t , was used to iden- 
tify the top quark in Ref. [ fl2| |. In this analysis, we require 
HjP < 240 GeV for those events satisfying the high-p^ 
single muon trigger to reduce the ti contribution. Sim- 
ilarly, since the low-px single muon trigger has a larger 
contribution from multijet events, we reject events with 



six or more jets (E T > 10 GeV) for that trigger. 

The resulting $ T distribution for data from all three 
triggers after all selection criteria have been applied is 
given in Fig. Also shown are the $ T distributions for 
MC events with M LQ = 100 and 200 GeV/c 2 . Requiring 
$ T > 35 GeV leaves two events. 

We consider background contributions from ti, inter- 
mediate vector boson, and multijet production of bb and 
cc. Top quark events have multiple b quarks and $ T , but 
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FIG. 2. The distributions of Ip T for data events after all 
other selection requirements have been applied. Shown in the 
insert are lfi T distributions for MC leptoquark events with 
leptoquark mass of 100 GeV/c 2 and 200 GeV/c 2 . 

TABLE I. Third generation scalar leptoquark acceptances 
for the three trigger channels and 95% C.L. limits on a ■ B 2 
for different Mlq- 

Acceptance (xlO -3 ) 



LQ mass 
(GeV/c 2 ) 


dimuon 


one muon 
low-pT 


one muon 
high-pr 


a-B 2 
limit (pb) 


50 


0.04 


0.95 


0.14 


144 


60 


0.08 


2.4 


0.35 


59 


80 


0.29 


6.0 


0.93 


22.6 


100 


0.40 


9.7 


1.9 


12.7 


125 


0.59 


14 


2.8 


8.9 


150 


1.2 


18 


3.2 


7.0 


200 


1.3 


24 


3.1 


6.0 


250 


1.9 


27 


3.7 


5.1 


300 


1.8 


31 


2.2 


5.4 



with additional, energetic jets. We use MC and our mea- 
sured ti production cross section jl3| to estimate that 
there are 1.4 ± 0.5 ti events in our sample. Intermediate 
vector boson events have Tf* T from W — > Iv or Z — > vv 
and muons near jets mimicking b quark decays when ei- 
ther a prompt muon overlaps a jet or a jet fragments 
into a muon via c quark or a ir/K decay. Using our 
measured W and Z boson production cross sections JIJ] 
yields 1.0 ± 0.4 W boson events and 0.1 ± 0.1 Z boson 
events in this sample. Hadronically-produced bb and cc 
events do not have energetic neutrinos and are effectively 
eliminated by the $ T and At/) cuts. Estimates of their 
contribution using data and MC are consistent with zero, 
and we conservatively assume this in our limit calcula- 
tion. Therefore, the total background is estimated to be 
2.5 ± 0.6 events. 

We calculate the detection efficiency for scalar lepto- 
quark signals using MC acceptances multiplied by muon 
trigger and reconstruction efficiencies obtained from data 
samples collected using test triggers. The acceptances for 
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TABLE II. Summary of systematic errors in terms of % 
error on the acceptance. 

Channel dimuon one muon one muon 







low-pT 


high-pr 


trigger 


6.5 


5.1 


5.1 


reconstruction 


5.1 


5.7 


4.2 


muon momentum resolution 


1 


1 


10 


jet energy scale 


4 


4 


2 


Ff, cut 


NA 


10 


6 


b — > (i fraction 


12 


6 


6 



different leptoquark masses are summarized in Table |. 
The use of a muon to tag b quark decays limits the ac- 
ceptance to values under 3.5%. Factors contributing to 
this limited acceptance for the low-px single muon chan- 
nel with M LQ = 100 GeV/c 2 include the muon branch- 
ing fraction (0.35), muon and jet kinematic requirements 
(0.35), and muon trigger and reconstruction efficiency 
(0.25). The requirements used to reduce the background 
($ T , Ar/>, Z^, F^) retain ss 40% of the leptoquark signal. 

We combine the three trigger channels to set limits. 
Errors on the acceptance are shown in Table EL Errors 
on trigger and reconstruction efficiencies are due to the 
statistical errors of the data used to calculate their values. 
Muon momentum resolution and jet energy scale errors 
are obtained from data and their impact on the accep- 
tance is determined using MC with Mlq — 100 GeV/c 2 . 
The error on the cut efficiency is estimated by compar- 
ing data events without Ifi T requirements to MC multijet 
events. The three trigger channels have different system- 
atic errors since their selection criteria and average muon 
Pt differ, but most errors are correlated. The total sys- 
tematic error, including correlations and MC statistics, 
on the combined acceptance varies between 12.5% and 
13.6% for different leptoquark masses. 

The 95% confidence level (C.L.) upper limits on a ■ B 2 
include the systematic acceptance uncertainty and a 5.3% 
uncertainty in the integrated luminosity. The resulting 
upper limits for scalar leptoquark pair production as a 
function of leptoquark mass are given in Table I and 
shown in Fig. |j| Also shown in Fig. || are theoretical 
cross sections for the production of scalar and vector lep- 
toquarks. The calculation of the scalar leptoquark cross 
section includes next-to-leading order diagrams and uses 
CTEQ4M parton distribution functions |l5|] . The theory 
band shown in the figure is determined using a renor- 
malization factor of [i = Mlq for the central value and 
fi = 2Mlq and Mlq/2 for the lower and upper bounds, 
respectively. The intersection of our limit curve with the 
lower edge of the theory band is at 94 GeV/c 2 . This is 
our 95% C.L. lower limit on the mass of a charge 1/3 
third generation scalar leptoquark (taking B = 1 since 
M LQ < m t ). 

Similarly, we set limits for the mass of vector lepto- 
quarks p6[. The vector leptoquark cross section has 
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FIG. 3. The 95% C.L. limit on a ■ B 2 (•) compared 
to theoretical predictions. The prediction for scalar lepto- 
quarks (solid) use /i = 2Mlq, Mlq and Mlq/2, while vector 
leptoquarks with minimal anomalous (dashed) or Yang-Mills 
(dot-dashed) coupling use fi = Mlq- 

been calculated in the leading order approximation using 
CTEQ4M parton distribution functions and /i = Mlq 
JItJ . Vector leptoquarks are assumed to be either funda- 
mental gauge bosons with Yang-Mills coupling or com- 
posite particles with anomalous coupling. For Yang-Mills 
type coupling, a mass limit of 216 GeV/c 2 is obtained for 
B = 1. If M l q > m t the rt mode is allowed. We con- 
sider the case in which the branching fraction to v T b and 
rt each would be 0.5 if the fermion masses could be ne- 
glected relative to Mlq- Taking into account phase space 
suppression factors [^8[ , we determine that Mlq > 209 
GeV/c 2 for Yang-Mills type vector leptoquarks for this 
B < 1 case. For anomalous coupling, we choose the cou- 
pling which yields the minimum pair production cross 
section. The intersection of our limit on a ■ B 2 with 
the theory curve gives Mlq > 148 GeV/c 2 for minimal 
anomalous vector coupling. 

In conclusion, we observe two events consistent with 
the final state vvbb compared to an expected 2.5 ± 0.6 
events from ti and W and Z boson production. We 
set limits on the mass of a charge 1/3 scalar or vector 
leptoquark. This result is independent of the coupling 
strength of a leptoquark to a third generation lepton and 
quark. 
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